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1. INTRODUCTION 

Future space exploration cannot do without special assisted robotic systems. Thanks to space robots, 
humans can conduct research and build space bases at large distances from earth. Unfortunately, such 
missions often must deal with a nondeterministic environment, which requires a space robot to constantly 
adapt to its changes. There are no robots yet that can quickly and accurately respond to significant changes in 
the working environment, so the presence of a human operator in the control loop is very important. In 
addition, when working over long distances, significant delays in the transmission of control and feedback 
signals occur. Therefore, the task of developing a bilateral space robot control system that is weakly 
dependent on the delay is very urgent. The master-slave system is most often used as a system for remote 
control of robot. The remote-control system has two sides: an operator side and a remote side (Figure 1) [1]. 
On the operator’s side, or in the control center, there are a human operator i), a master robot ii) and a 
television system, through which a human watch the remote side. On the remote side there are a slave robot 
111) with sensors and a vision system installed on it, and a working environment iv). Depending on the degree 
of decision-making by the robot, three main categories of remote-control architectures can be distinguished: 
direct, collaborative andsupervisory [2]. If the problem arises of manipulating objects that have connections, 
then when developing a remote-control system, it is necessary to include feedback on forces and moments in 
the loop. Force feedback robot remote control systems are called bilateral control systems. 
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Figure 1. The master-slave system 


The diagram shown in Figure 2 describes the connection and interaction between the elements of the 
robot’s bilateral control system. Here qn, qm» 4s» qe are the generalized coordinates of the human hand, the 
master robot, the slave and the environment, respectively. fp and f, are the forces transmitted by human and 
the environment, respectively. T and Tm are control signals for the slave and master robot. 

A significant problem in the development of bilateral control systems is the presence of a delay, 
which can lead to system instability. Researchers have tried to solve this problem with various methods, and 
as a result many different control schemes have been proposed. From the proposed methods, three main 
categories can be distinguished: methods based on passivity, methods based on predictive displays and 
methods of sliding control mode [3]. The method based on passivity is that the power developed by the 
working tool of the "slave" manipulator should not exceed the power developed by the hand of the human 
moving the tool of the master robot [4]. The second approach involves the use of predictive control. It is 
based on the use of computer models, which are used to predict the behavior of the slave robot and its 
external environment [5]. 

The third approach is based on the use of a sliding control mode [6]. The complexity of the practical 
implementation of this approach is due to the need for the operation of the control equipment and the 
mechanical part of the robot in very difficult modes of frequently changing control sign. This leads to the 
appearance of large accelerations of structural elements and large reactive forces. 

The above methods can ensure the stability of the bilateral system with a time delay of no more than 3 
seconds [7], which significantly limits the distance at which a space robot can be sent. In this regard, an urgent 
question arises of constructing such a structure of bilateral control, which would provide the possibility of 
remote control over long distances. As an approach to solving this issue, a professor at St. Petersburg State 
University F. Kulakov proposed a method of remote control of a robot by teaching future actions [8]-[14]. 





Human Master 
Operator Robot lg. Environment 


Figure 2. Structure of bilateral master-slave system 


2. METHOD 
In this approach to control, the remote working environment is copied in a virtual or full-scale form 
in the control center and the human operator interacts in the necessary way with the objects of the model 
environment, after which the space robot works out the same actions at the objects of the remote 
environment, while adapting to its possible changes. The process of control of a space robot using the 
teaching method can be divided into 4 stages (Figure 3). 
— Stage 1. At the first stage, the space robot, using sensors, collects the necessary information about the 
working environment and transmits it to the control center. 
— Stage 2. At the second stage, a model of the remote environment is formed based on the received data. 
The model can be made in the form of full-scale models or built on a computer. 
— Stage 3. The third stage, which takes place at the command center on earth, can be performed in two 
different ways, depending on the type of remote environment model. If the working environment is 
presented in the form of a full-scale model, then interaction with it occurs according to the following 
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principle: the human operator, interacting with the master robot, sets the required position, and orientation 
of the tool, the instrument of this robot must accurately interpret the instrument of the space robot. Thus, 
the operator interacts with the objects of the full-scale model, as in the traditional method of bilateral 
control. Instrument motions are memorized and transmitted to the space robot. If necessary, a space robot 
can be displayed to a human operator in augmented reality glasses. If it is not possible to build a full-scale 
model, then you can use the virtual model of the working environment. In this case, at the third stage, the 
human operator, observing the virtual environment through augmented reality glasses, interacts with the 
objects of this environment using a robot-gloves, and thereby fulfills the necessary operations. In both 
cases, if necessary, the resulting algorithm is tested on a computer model of a space robot and a working 
medium, with its objects being artificially displaced. If the test is successful, the algorithm is passed to the 
slave robot. Otherwise, the necessary changes are made to the algorithm and the test is conducted again. 

— Stage 4. At the last stage, the space robot performs the obtained algorithm of actions, while adapting to 
possible changes in the working environment, and sends a report on the work done to the control center. 
If the control center confirms the success and completeness of the mission, then the stages are terminated. 
Otherwise, the cycle is repeated until the required operation is completed. 





Stage 4 


Figure 3. Process of control space robot by teaching method 


2.1. Mathematical modeling of the executive system 

Any robotic system requires preliminary mathematical modeling. Therefore, to test the operability 
of the proposed remote-control system for a space robot, it is necessary to simulate its elements and their 
interaction. The system under consideration consists of five key elements: a human operator, a master robot, 
a space robot model, a real space robot, and a working environment. Since the master robot is an instrument 
that transmits motion and force data between the space robot model and the human operator, mathematical 
modeling of such an object is of no interest. In this regard, the system "human operator-master robot" will be 
presented as a block with output and input data. The working environment will be presented in a similar 
block since it will be full-scale and data about it will be read during the direct development of the necessary 
operations. Therefore, it is necessary to build only a model of a space robot and a model of a ground robot, 
the grip of which will be identical to the tool of the space robot. In Figure 4 elements of modeling which need 
to be performed are displayed. Kinematic modeling of robots is necessary to determine their kinematic 
characteristics over time. Information about the position of the characteristic points of the robot will be useful 
for graphical display of its motion. The solution of the inverse kinematics problem will make it possible to 
determine the values of the generalized coordinates that provide a given position and orientation of the tool in 
absolute space. And information about speeds is necessary when building a dynamic model. 
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Figure 4. Elements of a remote-control system for a space robot using the teaching method 


2.2. Kinematic schemes 

Let us build a kinematic diagram of a space robot, which in general will be a body with a n-link 
manipulator installed on it (Figure 5). Thus, the robot will consist of n + 1 solids. Let’s associate a 
corresponding coordinate system with each element of the robot. We will assume that each link can linearly 
move and rotate about the corresponding axis with respect to the previous body, that is, it has at least two 
generalized coordinates-linear and rotational. In this case, the robot will have (2n + 6)-th generalized 
coordinates. The first six coordinates correspond to the position and orientation of the robot body, the next 
2n coordinates are responsible for the position and orientation of each link of the manipulator. Generalized 
coordinates q1, q2,q3 determine the rotation of the body relative to the axes OgoXo,OoYo and Op Zo, 
respectively, and q4, qs, qę are responsible for moving the center of mass of the body along the same axes. 
Coordinate q¢42;-1 corresponds to the rotation of the i-th link, and q,¢,3;-to its linear movement. The rotation 
and movement of the i th link can occur along any of the three axes of its coordinate system O;x;,O;y; or 
O;z;. If the robot is intended to carry out work in the orbit of a celestial body, then the body of the robot will 
be free and all its generalized coordinates will take place. If the robot operates on the surface of a celestial 
body, then the generalized coordinates q4, q2 and qd, will be constant. Let us move on to building a kinematic 
scheme of a ground robot, the instrument of which will display the instrument of the space robot. As such a 
robot, let us consider the industrial manipulator fuji automatic numerical control (FANUC) M20-iA [15], [16], 
which is actively used at the Faculty of Applied Mathematics-Control Processes of St. Petersburg State 
University. Let us associate a corresponding coordinate system with each manipulator body and denote 
generalized coordinates (Figure 6). Based on the constructed schemes, it is possible to solve forward and 
inverse kinematics problems, forward instantaneous kinematics problem, presenting robots as a set of 
interconnected coordinate systems. 


Oxy KA O72, OX.Y.Z; OXYZ 







OXY,2, 


OXY 4, 





Figure 5. Generalized kinematic scheme of a space Figure 6. Kinematic scheme of the manipulator 
robot FANUC M20-i1A 


2.3. Forward instantaneous kinematics problem 

To build a dynamic model of a robot, it is necessary first to find the dependence of the linear and 
angular velocities of the links on the generalized velocities. The search will be carried out using the recurrent 
formulas presented in [17]. Let us solve the forward instantaneous kinematics problem for a space free-flying 
robot. The angular velocity of a rigid body is directed along the axis of rotation. The direction of the angular 
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velocity vector depends on the direction of rotation. If we use the theorem on the addition of angular 
velocities in a complex motion, then the relationship between the vector of the angular velocity of the i - th 
rigid body and the vector of generalized velocities, expressed in the absolute coordinate system, can be found 
by the formula [18]: 


Twy] = EA- [41,42 --»Go]’, where ‘OQ = EIR; 104 "I, 


a 
wz |, 


here the matrix tI, defines the direction of the axis of rotation of the i - th body. 

The space robot in question consists of four bodies. The projections of the angular velocity of the 
robot body do not depend on the angular velocities of its links. Consequently, in the absolute coordinate 
system, the angular velocity of the body will be expressed through the vector of generalized velocities as 
follows; 


Wy qı Cq2Cq3 — Sq2Cq15q3  Sq3Cq2 + Cq3CqıSq2 Sqı5q2 
“Wy = "Ro: |2], “Ro = |—Cq35q2 — Sq3CQ1Cq2 —Sd25q3 +Cq2Cq1Cq3 S$q,Cq2}, 
"Ww, q3 Sq3544 —Cq3544 Cay 








in this case 


Cq2Cq3 —Sq2Cqi5q3  Sq3Cq2 + Cq3Cqi$q. SqiSq2 9 O D0 0 0O O 
—Cq3Sqd2 —Sq3Cq,Cqd2 —Sq25q3+Cq2CqiCq3 SqiCq. O 09 0 0O 0O OQ 
Sq35Q4 —Cq3Sq, Cq 0 0 0 0 0 


990 = 





Then for the Ist link the angular velocity will be equal to 
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here 
1 0 0 0 0 0 0 0 0 1 0 0 
°R, =|0 Cq; —Sq7], Inf=!0 0 0 0 0 0 0 0 0 
0 Sq, Cqy 0 0 0 0 0 0 0 0 0 
then 
1w twp œz 00 01 0 0 
O) = 1w W22 Aa 0 0 0 0 0 0 j 
1w; 4W32 w3, 00 0 0 0 0 


1w11 = Cq2Cq3, *W12 = —Cq1Sq3 — Cq3Sq1Sq2, *W13 = SqŅ1Sq3 — Cq1Cq3Sq2, 
*W21 = Cq2C 47543 — S257, *W22 = Cq7(Cq1Cq3 — S41542593) — Cq2Sq1 S07, 
*W23 = —Cq7(Cq3Sq1 + Cq1Sq25q3) — CQ1C 42547, 

*W31 = Cq7Sq2 + Cq25q3547, *W32 = Sq7(Cq1Cq3 — $41542543) + Cq2Cq7Sq1, 


*wW33 = Cq,Cq2Cq7 — Sq7(Cq3Sq1 + CqySq25q3). 


2w °w °w 0 0O 0O Cqg O O0 
20 = ER w22 W23 0 0 0 Sdg 0 0 j 
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2.4. Linear velocities of the space robot 

Let us find the dependence of the linear velocities of the characteristic points of the robot on the 
vector of generalized velocities. The generalized coordinates q4, qs and qę of its body are responsible for the 
linear movements of the robot. In addition, when the links and the base of the robot rotate, the characteristic 
points will have linear velocities perpendicular to the direction of the angular velocity vector and calculated 
by the Euler formula. The relationship between linear and generalized velocities, with respect to the i-th 
characteristic point (“0”-center of mass of the body, “1, 2, 3”—beginning of 1, 2, 3 links, “4”—characteristic 
point of the tool), is determined in the following form (using the theorem on the addition of velocities in a 
complex motion and the formula for the distribution of velocities in a solid [19], [20]): 


a 
Vx 
R = ty ° [41, d2» aadal where iy = HAR: ° i-1y7 + SER, : Gis A] Q, 
a 
Vz i 


here the skew-symmetric matrix L;_, determines the distance from the i - th point to the axis of rotation of 
the previous body. 
With respect to the Oth characteristic point: 


a 








Vy q4 
“Vy| = "Ro: |ġs |. Therefore 
KA de 
0 0 0 Cq2Cq3—Sq2Cq15q3 Sq3Cq2 + Cq3Cqi5q2 Sq 0 0 0 
°V=10 0 0 —Cq3Sq2 — Sq3CqıCq2 —Sq2Sq3+Cq2CqiCq3 SqiCqz O 0 OJ. 
0 0 0 Sq3Sq, —C q5 q1 Cq 0 00 
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similarly for the 2nd, 3rd and 4th characteristic points: 
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2.5. Forward dynamics problem 

For further investigation of the operability of the system of bilateral control by the teaching 
method, it is necessary to construct a dynamic model of a space free-flying robot and a ground manipulator 
FANUC [19], [20]. The forward dynamics problem is to search for forces and moments for a given motion of 
the system and the mass—inertial characteristics of its elements. Let us solve the forward dynamics problem 
for space and ground robots. The equations of motion for robots will be sought in the form of the Lagrange 
equation of the second kind. 
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The kinetic energy of the system is equal to the sum of the kinetic energies of its elements. Let’s 
find the kinetic energy for a space free-flying robot (Figure 7). The robot consists of four bodies-a body and 
three links, therefore, having calculated the value of the kinetic energy for each body, we will find the kinetic 
energy of the entire system. 


Model Environment 


Real inwiraninent The Last Link of the Manipulator 
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Figure 7. Search for characteristic points of the environment using laser rangefinders 


T = i-o Ti. 
The kinetic energy of a rigid body in the general case of motion is equal to: 
T; = Xi BX), (1) 
where xX is the Plücker coordinates [21] of the velocity of the body (1), and the matrix B characterizes the 


mass—inertial characteristics. Matrices B for rigid bodies of a space robot will have the following form (we 
neglect the moments of inertia with respect to the axes of the links): 


Jo 0 0 0 0 0 I} 0 0 0 0 miyf 
0 J 0 0 0 0 0 0 0 0 0 0 
B, = 0 0 J 0 0 0 B= 0 0 TP i —miy; 0 0 i=123, 
0 00m 0 0 0 0 -—miyz mi 0 0 
0 0 00 m0 0 0 0 0 m; 0 
0 0 0 0 0 m myf 0 0 0 0 mi 


here yf is the coordinate of the center of mass of the link along the Oy axis (in the coordinate system of the 
i-th link). In this case, the kinetic energy of the entire space robot will be found by the formula: 


Bo O 0 0 
0 B 0 0 

gae -T poe = 1 
T; = x’ Bx, where B 0 0 B, 0 
0 0 O Bs; 


For a ground robot, the matrices B will have the following form: 
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I 0 0 0 —m,zf 0 
0 I? 0 mzf 0 0 
0 0 0 0 0 0 
B. = = 1,2 
ilo miz 0m 0 a (ee 
—m;z; 0 0 0 Mi 0 
0 0 00 0 m; 
A 0 0 0 0 mi yf 
0 0 0 0 0 0 
_ {0 0 TP —myf 0 0 S 
B= 0 —myl mi d. Gg (per eee 
0 00 0 m; 0 
my; O 0 0 0 mi 


here zř is the coordinate of the center of mass of the link along the Oz axis (in the coordinate system of the i 
— th link), and y; is the coordinate of the center of mass of the link along the Oy axis. If we substitute (1), 
then we can represent the kinetic energy of the system in the form: 


T ==" A(q)q, where A = CTBC. 


now we substitute the obtained expression for the kinetic energy through the generalized velocities into the 
Lagrange equation of the second kind: 


a ofza"aa]  afza"aa] _ 


a [z4 Ad 
dt dq ðq í ; 


Q, 2 = A(a)a, 5 ADA = ACA + ACG, 


in the case of a 9-degree of mobility of the space robot (for the ground robot there will be 6 components) 


9 0041(q) . 9 9040(q) . 
E oe De ni 

i= ðqj 0qj alsa aq] a.y pa l 
9 0a91(q) . 9 dao9(Q) . á ðq 2 ðq 
jJ=1 aqj J in jJ=1 dq; J 

x? 0a141(q) 9 0049(q) 

j=1 } n.a j=1 

aa@)_ | OO 

dq 9 0a91(q) 9 da90(q)| 


Then the equations of motion in general form can be written as follows (for a ground robot, instead 
of 9 there will be 6) 


A(q)q + Ys=1 [4° Ds(a)dles = Q, (1) 


where D,;(q) = (d?-(q)),i,t,s = 1,...,n, 











1 (0a; Oats Ait 
ds zi ~( is _ 9a; ) 
t(D) = 3 Voar + aq bas?" 


where e, is a unit vector a column whose unit is in the s - th row. 

Substituting into (2) the desired law of variation of generalized coordinates, we find generalized 
forces that provide the desired motion of the robot. Hence, (2) is the solution to the forward dynamics 
problem. 


3. ADAPTING TO ENVIRONMENTAL CHANGES 

For the robot’s working tool to successfully perform an operation with an object, it is necessary that 
in the process of implementing the operation the position of the object relative to the working tool, uniquely 
determined by the position of its characteristic points, as well as the magnitude of the force of interaction 
between the tool and the object, are identical to the forces and position of their models in the process of 
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studying. Let us consider an adaptation method based on the search for characteristic points of objects using 
laser rangefinders, which are necessary to calculate the difference between the real position and rotation of 
the body from its model position. The range finders will be installed at the end of the last link of the 
manipulator, rigidly connected to the body of the wrist force-torque sensor (Figure 7). The movable part of 
the sensor structure is usually fastened to the working tool of the manipulator, which allows the gripper to 
slightly displace relative to the last link due to deformation of the elastic structure of the sensor under the 
action of the force applied to the gripper when it interacts with environmental objects. Thus, the positions of 
the characteristic points of environmental objects presented in the coordinate system of the last link (sensor 
body) slightly differ from their position in the gripper coordinate system and coincide with it in the absence 
of elastic deformation of the sensor, that is, in the absence of interaction of the gripper with objects of the 
external environment. When using laser rangefinders, the characteristic points are the points of reflection of 
the laser beam from the surface of the models of environmental objects obtained during the training process, 
and the points of reflection of the beam from real objects obtained during the implementation of the required 
action. Each point corresponds to three components of the position vector in the coordinate system of the last 
link of the manipulator. The most common difference between the real external environment and its model is 
not the difference in spatial configurations of objects, but basically only in the relative displacement and 
rotation relative to each other. Therefore, to achieve the position of the working tool relative to the surface of 
the external environment, identical to their relative model position, it is necessary to rotate and shift the 
gripper accordingly. Figure 7 explains this. On it, for convenience of perception, not a three-dimensional, but 
a two-dimensional case of the external environment is presented. For the position of the gripper relative to 
the real surface to be identical to the position of its model relative to the model surface, it is necessary that 
the position vectors of at least two points are equal, that is, x = X' i = 1,2,3, .... For the practical 
implementation of the effect of combining characteristic points corresponding to each other in the control 
process, it is advisable to use in the control law for the local control system of a space robot a term that is a 
function of the value of the mismatch between the desired XÍ and the current XÍ vectors of position of 
characteristic points of the external environment. And to maintain the vector of force of interaction of the 
working tool with objects of the external environment Q close to the desired Qf obtained during training, it is 
necessary to use in the control law, in addition to the above, the term that is a function the magnitude of the 
mismatch between the vectors Qf and Q. The law is as follows: 


LT 
1 ax’ i i d 
U=+yt, (5) K,(X4, — XÏ) + Ko (Q? — Q), 
where Qf and Q are vectors of the desired and current values of the forces of interaction of the gripper with 
the object of the external environment. 


4. RESULTS AND DISCUSSION 

To verify the results, we will carry out computer modeling in the Matlab kinematics package. The 
package is broken up into functions that deal primarily with homogeneous transforms and their Lie algebra, 
and a set of functions for interacting with serial link kinematic structures. There are also quite a few functions 
for generating nice plots and animations of the results. Included is an html documentation page that lists all 
the functions, and each function provides its own documentation. First, we will model the Kinematic of a 
free-flying space robot, then after we will model the dynamic where we represent the solution of the forward 
dynamics problem. 


4.1. Kinematic modeling 
Let the following initial and final values of the generalized coordinates of a free-flying space 
robot [22], [23] be given (Figure 7): 


qQo=[0 0 -n/2 0 2 1 0 0 Off, 
qg =[-7/6 0 w/4 0 -2 5 2n/3 -n/2 Of 
let us introduce the following dependence of the change in the generalized coordinates on time 


: 1 A 
qi(t) = Åq} - 42), i=1,...,9, where fio A(t)dt = 1, Let us Å(Ct) =< 


then we get the following motion of a free-flying space robot (Figure 8). 
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Let us present a computer simulation of the forward kinematics problem for the FANUC ground 
manipulator [24], [25]. Take the following coordinates as the initial and final values: 


qo = 0, qg = [3a/2 —rn/8 m/3 nm/6 -n/2 1/4]! 


the dependence of the generalized coordinates on time will be similar. In this case, the motion of the FANUC 
manipulator (Figure 6) is shown in Figure 9. 

We will solve the inverse kinematics problem simultaneously for both ground and space robots, 
since their tools must be the same for bilateral control. The space robot will be virtual, so it will be shown 
with a dashed line. As input data, we take two positions of the tool, into which the robots should come from 
their initial state: 


Xa = [1.5 2 3 7/3 0 a/12]',xg,=[-1 -1 4 2/3 0 —r/12]f, 


here the first three components of the vector are responsible for the coordinates of the characteristic point of 
the tool in absolute space, and the last three for the orientation of the tool. The motion of robots, as in the 
case of the forward kinematics problem, will also be uniform. Figure 10 illustrates the positions of the robots 
at the initial moment of time, as well as in the given positions of the tool. 





Figure 8. Solution of the forward kinematics Figure 9. Solution of the forward kinematics problem 
problem for the space robot for the ground robot 





Figure 10. Solution of the inverse kinematics problem for the space and ground robot 


4.2. Dynamic modeling 

We represent the solution of the forward dynamics problem. The task is to search for generalized 
forces that provide a given law of change of generalized coordinates. Let us give a solution to the problem for 
the free-flying space robot. Let us set the following law of change of generalized coordinates: 
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qo=0,q,=[0 0 m/2 6 -6 4 n/4 n/3 —-n/2]', 
q(t) = A(t) (qi — q?), i= 1,...,9, 


where: 


1: 5; : 
fi A(t)dt = 1, A(t) = “ sin? ZT = ti — to. 


the generalized forces corresponding to this motion are shown in Figure 11. 


Here is a solution to the problem for a ground robot FANUC. Let us simulate the following law of 
change of generalized coordinates: 


qo = 0, qk = [37/2 -7/8 2/3 m/6 -n/2 n/A", q(t) =A -qP i= 1.9, 
where: 
T 4 ; 
fi A(t)dt = 1, A(t) = sin? =, = ti = to. 


the robot will move in the same way as in Figure 12, but smoother with zero start and end velocities. The 
generalized forces for this case are shown in Figure 13. 








Figure 12. The law of change of generalized forces of the space robot 
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Figure 13. The law of change of generalized forces of the ground robot 


5. CONCLUSION 

The result of the study is the construction of a kinematic and dynamic model of space and ground 
robots, which are the foundation for future studies of the bilateral control system by showing future actions. 
In the future, it is planned to optimize the algorithm for solving the forward dynamics problem, to identify 
the control vector from the equations of dynamics, to solve the inverse dynamics problem for a given control 
vector, to check the operability of the control law for adapting the robot to environmental changes, and to 
study the dynamics of the behavior of a space and ground robot during interaction robot tool with work 
environment objects. 
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